The 4-D approach to visual control of autonomous systems

Development of a 4-D approach to dynamic machine vision is described. Core elements of this method are spatio-temporal models oriented towards objects and laws of perspective projection in a foward mode. Integration of multi-sensory measurement data was achieved through spatio-temporal models as invariants for object recognition. Situation assessment and long term predictions were allowed through maintenance of a symbolic 4-D image of processes involving objects. Behavioral capabilities were easily realized by state feedback and feed-foward control

May a Pair of ‘Eyes’ Be Optimal for Vehicles Too?

Following a very brief look at the human vision system, an extended summary of our own elemental steps towards future vision systems for ground vehicles is given, leading to the proposal made in the main part. The question is raised of why the predominant solution in biological vision systems, namely pairs of eyes (very often multi-focal and gaze-controllable), has not been found in technical systems up to now, though it may be a useful or even optimal solution for vehicles too. Two potential candidates with perception capabilities closer to the human sense of vision are discussed in some detail: one with all cameras mounted in a fixed way onto the body of the vehicle, and one with a multi-focal gaze-controllable set of cameras. Such compact systems are considered advantageous for many types of vehicles if a human level of performance in dynamic real-time vision and detailed scene understanding is the goal. Increasingly general realizations of these types of vision systems may take all of the 21st century to be developed. The big challenge for such systems with the capability of learning while seeing will be more on the software side than on the hardware side required for sensing and computing

A General Cognitive System Architecture Based on Dynamic Vision for Motion Control

Animation of spatio-temporal generic models for 3-D shape and motion of objects and subjects, based on feature sets evaluated in parallel from several image streams, is considered to be the core of dynamic vision. Subjects are a special kind of objects capable of sensing environmental parameters and of initiating own actions in combination with stored knowledge. Object / subject recognition and scene understanding are achieved on different levels and scales. Multiple objects are tracked individually in the image streams for perceiving their actual state ('here and now'). By analyzing motion of all relevant objects / subjects over a larger time scale on the level of state variables in the 'scene tree representation' known from computer graphics, the situation with respect to decision taking is assessed. Behavioral capabilities of subjects are represented explicitly on an abstract level for characterizing their potential behaviors. These are generated by stereotypical feed-forward and feedback control applications on a separate systems dynamics level with corresponding methods close to the actuator hardware. This dual representation on an abstract level (for decision making) and on the implementation level allows for flexibility and easy adaptation or extension. Results are shown for road vehicle guidance based on three cameras on a gaze control platform

Three-Stage Visual Perception for Vertebrate-type Dynamic Machine Vision

Abstract: Efficient real-time visual perception in civilized natural environments (e.g. road networks) has to take advantage of foveal – peripheral differentiation for data economy and of active gaze control for a number of benefits. 1. Inertial gaze stabilization considerably alleviates the evaluation of image sequences taken by cameras with stronger tele-lenses; it allows a reduction in angular disturbances from rough ground by at least an order of magnitude with simple negative angular rate data feedback. 2. Visual tracking of fast moving objects reduces motion blur for these objects. – 3. In the near range, a large field of view is mandatory, however, only coarse angular resolution is sufficient; with a field of view (f.o.v.)> ~ 100°, both the region in front of and to the side of the vehicle may be viewed simultaneously. For own behavior decision, motion behaviors of objects both in the wide f.o.v. nearby and in several regions of special interest further away have to be understood in conjunction. In order to achieve this efficiently, three distinct visual processes with specific knowledge bases have to be employed in a consecutive way. In the wide f.o.v., bottom-up feature extraction has to answer the question: ‘Is there anything of special interest? ’ The corresponding feature extraction operators are domain-specific. On initialization, they have to give indications of objects of interest all over the image. Stable feature aggregations over several cycles have to trigger object hypotheses for the second stage; these regions may then be discarded for stage 1. Stage 2 works on single objects, however, on multiple of these in parallel. When looking almost parallel t